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(54) A data dictionary manager for maintaining an active data dictionary. 

(g) The data dictionary manager (85) takes « 
advantage of the computer system's journal ing 
capability enhanced to allow users and appli- 
cation programs to manipulate system objects 
(65) without the use of the data dictionary's 
built-in utilities. As used here, journal ing 
capability is an internal tracking facility which 
exists in a somewhat limited form on many 
computer systems. Typical journal ing mechan- 
isms (38) maintain a repository of information 
about some of the activities that have taken 
place on the computer system. The information 
is usually stored in a record called an audit 
journal (75). Since many computer systems 
have limited journal! ng mechanisms already in 
place, these mechanisms can be enhanced to 
add the ability to record information about 
changes to system objects. Examples of system 
object changes included in the audit journal are 
deletes, creates, renames, and moves. Once the 
information has been logged in the audit jour- 
nal, the data dictionary manager retrieves the 
information from the audit journal and ensures 
that the changes are accurately reflected in the 
data dictionary (90). Since the audit journal can 
be made accessible to several processes, it is 
possible to have different instances of the data 
dictionary manager responsible for different 
data dictionaries. Alternatively, it is possible to 
have a single data dictionary manager that is 
responsible for all of the computer system's 
data dictionaries. 
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BACKGROUND OF THE INVENTION 

This invention involves the management of a data 
dictionary in a computer system and the system ob- 
jects that it tracks. A data dictionary is defined in the 5 
IBM Dictionary of Computing as "[a] centralized repo- 
sitory of information about data such as meaning, re- 
lationships to other data, origin, usage, and format." 
As such, a data dictionary is not unlike the standard 
dictionary of a spoken language (e.g., Webster's Col- 1 o 
(egiate Dictionary). Both the data dictionary and the 
standard dictionary contain information about certain 
items. The difference between the two concepts is 
the type of items that are involved. The items of inter- 
est in the standard dictionary are words; whereas, the 15 
items of interest in the data dictionary are called ob- 
jects. An object can be a particular file or record, an 
instance of software (i.e., a program), or some other 
important system resource. The data dictionary is 
used by the computer system to keep track of the sta- 20 
tus of these important system resources. The benefit 
of using a data dictionary is, in a word, control. Ideally, 
the exact status of every system resource is known 
at all times. With this control, however, is the associ- 
ated problem of ensuring that the data dictionary has 25 
current information regarding every object. To analo- 
gize this situation to that of the standard dictionary, 
one need only consider the importance of having an 
up to date dictionary. Newer words and/or definitions 
are often lacking in older dictionaries. As the diction- 30 
ary becomes more and more out of date, it becomes 
less and less valuable. The problem is amplified in 
data dictionaries because, as opposed to words, the 
computer system's objects are constantly being modi- 
fied. 35 

To deal with this problem, typical data dictionary 
designs include various built-in mechanisms that per- 
form both the modification of the objects and the as- 
sociated change to the data dictionary. That is, when 
a user or an application program requires that a 40 
change be made to a particular object, an interface to 
the data dictionary is used to make the change. Then, 
an internal mechanism of the data dictionary will au- 
tomatically make the associated dictionary changes. 
The problem with these current implementations is 45 
that to be kept active, they require that all modifica- 
tions be carried out through their built-in utilities. This 
results in decreased flexibility. Application programs 
are limited to potentially inadequate mechanisms to 
access various system resources. Further, since data so 
dictionary utilities must themselves gain access to 
system resources through operating system utilities, 
overall system performance is adversely affected. 

Another problem inherent in existing data diction- 
ary designs is the exposure to massive inaccuracies. 55 
Since existing data dictionary designs require that all 
changes to dictionary defined objects be made 
through built-in utilities, they are not aware of 
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changes which occur outside of their sphere of con- 
trol. This means that when changes of this type do oc- 
cur, the data dictionary cannot be depended upon as 
an accurate reflection of the state of system objects. 
This problem is amplified when one considers that in 
some cases the only solution is a time consuming sys- 
tem reload which in and of itself may result in the loss 
of data. 

SUMMARY OF THE INVENTION 

It is a principal object of this invention to provide 
an enhanced method and apparatus for the manage- 
ment of a data dictionary. 

It is another object of this invention to provide a 
method and apparatus for managing a data dictionary 
which provides for more synchronization between a 
data dictionary and the existing system environment 
by tracking changes to system objects regardless of 
how the changes are effectuated. 

It is still another object of this invention to provide 
an enhanced dictionary management method and ap- 
paratus which can be adjusted to provide the user 
with more flexibility in choosing the degree of syn- 
chronization. 

It is still another object of this invention to provide 
an enhanced dictionary management method and ap- 
paratus for improving the overall integrity of the data 
dictionary by alerting the system administrator of any 
threats to the data dictionary's integrity. 

These and other objects are accomplished by the 
data dictionary manager disclosed herein. 

The data dictionary manager takes advantage of 
the computer system's journal ing capability en- 
hanced to allow users and application programs to 
manipulate system objects without the use of the data 
dictionary's built-in utilities. As used here, journaling 
capability is an internal tracking facility which exists 
in a somewhat limited form on many computer sys- 
tems. Typical journaling mechanisms maintain a re- 
pository of information about some of the activities 
that have taken place on the computer system. The 
information is usually stored in a record called an au- 
dit journal. Since many computer systems have limit- 
ed journaling mechanisms already in place, these 
mechanisms can be enhanced to add the ability to re- 
cord information about changes to system objects. 
Examples of system object changes included in the 
audit journal are deletes, creates, renames, and 
moves. 

Once the information has been logged in the au- 
dit journal, the data dictionary manager retrieves the 
information from the audit journal and ensures that 
the changes are accurately reflected in the data dic- 
tionary. Since the audit journal can be made acces- 
sible to several processes, it is possible to have dif- 
ferent instances of the data dictionary manager re- 
sponsible for different data dictionaries. Alternatively, 



2 



EP 0 550 372 A2 



it is possible to have a single data dictionary manager 
that is responsible for all of the computer system's 
data dictionaries. 

Since information about changes to system ob- 
jects is recorded at the operating system level, the 5 
present invention does not share the inflexibility prob- 
lems of existing data dictionary implementations. 
Modifications to system objects can be made without 
the use of built-in utilities and without the loss of syn- 
chronization between the data dictionary and the 10 
state of system objects. To realize this flexibility, the 
recorded information includes any and all changes 
that have been made to system objects. 

Although the information could be stored in an- 
other manner, the existing audit journal provides the 15 
easiest and most cost effective repository for the in- 
formation. The invention provides further flexibility by 
allowing the user (system administrator) of the data 
dictionary manager to control exactly how accurately 
the data dictionary reflects the current state of sys- 20 
tern objects. This control is achieved by making the 
data dictionary manager responsive to both user 
stimulus and adjustable system timers. 

This is beneficial in that it allows the system ad- 
ministrator to adjust the data dictionary manager to 25 
best handle changing system requirements. 

The data dictionary manager is further responsi- 
ble for alerting the system administrator of potential 
problems with the configuration of the system's jour- 
nal ing mechanism. The data dictionary manager ac- 30 
complishes this by monitoring system values to detect 
any system changes that may impact the integrity of 
the. data dictionary. This capability allows the system 
administrator to correct any problems before they be- 
come catastrophic. 35 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows the computer system of the prefer- 
red embodiment 40 

Fig. 2 shows how the various entities of a com- 
puter system interrelate to keep the data dictionary 
up to date with system activities according to the pre- 
ferred embodiment 

Fig. 3 shows how the data dictionary manager in- 45 
teracts with the dictionary services and the audit 
journal to keep the data dictionary up to date with the 
state of system objects according to the preferred em- 
bodiment 

Fig. 4 is a flow diagram of the operation of the 50 
data dictionary manager according to the preferred 
embodiment 



DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

Fig. 1 shows a block diagram of a typical comput- 
er system suitable for operation of the present inven- 
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tion. In the preferred embodiment computer system 
10 is an IBM Application System/400 midrange com- 
puter; however, other systems such as personal com- 
puters and mainframe computer systems could also 
be used. Contained within computer system 10 are 
data storage 15, CPU 20, memory 25, and terminal 
interface 27. Data storage 15, as secondary memory, 
may be a magnetic disk drive, an optical storage de- 
vice, or a combination of different storage devices. 
CPU 20 is responsible for executing the software pro- 
grams that have been loaded into memory 25. Termi- 
nal interface 27 allows developers and users to com- 
municate with computer system 10, normally through 
a programmable workstation. 

Data storage 15 contains application programs 
30, system object manipulation interface 35, audit 
journal 75, and operating system 40. Operating sys- 
tem 40 further contains journal ing mechanism 38. 
While storage 15 is shown as a monolithic entity, it 
should be understood that it may comprise a variety 
of devices, and that all programs and files shown will 
not necessarily be contained in any one device. For 
example, portions of application programs 30 and op- 
erating system 40 will typically be loaded into primary 
memory 25 to execute, while source data files will typ- 
ically be stored on magnetic or optical disk storage de- 
vices. 

Fig. 2 shows how the separate conceptual enti- 
ties of computer system 1 0 work together to maintain 
active data dictionary 90. As mentioned above, one 
purpose of the invention is to keep data dictionary 90 
up to date with changes to system objects without 
necessarily involving dictionary services 45. This ca- 
pability is enabled by the journaling mechanism 38. 
As mentioned above, journaling mechanism 38, as a 
part of operating system 40, is used by computer sys- 
tems to track (i.e., keep a journal of) various system 
activities. Since journaling mechanism 38 is a "low 
level" program, all changes made by higher level rou- 
tines are know to it Information about selected 
changes are sequentially recorded in audit journal 75. 
Audit journal 75 is shown in Fig. 2B. Audit journal 75 
is made up of a series of entries. Each entry has a 
time 300, a date 310, a journal entry type 320, an ob- 
ject ID 330, a user ID 340, and data 350. Time 300 and 
date 310 indicate the time in which journaling mech- 
anism 38 logged the particular entry. Journal entry 
type 320 indicates what type of entry is involved (e.g., 
DO is a delete entry). The journal entry types of the 
preferred embodiment are shown in Fig. 2C. Object 
ID 330 identifies the object for which the particular 
entry pertains. User ID 340 identifies the user or task 
which made the subject change. Data 350 includes in- 
formation which is particular to the subject change. 
For example, for a create entry (CO) data 350 would 
include all the information involved in the creation of 
the object identified by object ID 2211 . 

In some cases, an existing journaling mechanism 
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may need to be augmented to provide additional infor- 
mation about changes to system objects. In the pre- 
ferred embodiment, journaling mechanism 38 is first 
initialized to ensure that all necessary activities are 
tracked. The particular activities that are tracked in- 
clude object deletes, object creates, object moves, 
object renames and object restores. In addition, jour- 
naling mechanism 38 tracks changes that occur to its 
configuration. This allows data dictionary manager 85 
to take appropriate action when a necessary activity 
is no longer being tracked by journaling mechanism 
38. 

Logical blocks 50, 52, and 54 depict three possi- 
ble classes of software entities that can co-exist with 
dictionary manager 85. Each block represents a pro- 
gram or utility that is independently capable of per- 
forming changes to system objects through direct in- 
teraction with operating system 40. Since operating 
system 40 is always used to effectuate the change, 
journaling mechanism 38 is able to maintain an accu- 
rate record of all the changes by placing change en- 
tries in audit journal 75. For example, if application 
programs 54 need to create a system object (e.g., a 
queue), they simply invoke the appropriate routine 
within operating system 40. Journaling mechanism 
38, aware that an object has been created, records 
the occurrence of the change in audit journal 75. It is 
then the responsibility of data dictionary manager 85 
to retrieve the information from audit journal 75 and 
update data dictionary 90 accordingly. In this manner, 
data dictionary 90 is kept abreast of the changes in 
the system environment regardless of how they are 
effectuated. 

Fig. 3 shows the interaction of data dictionary 
manager 85, dictionary services 45, and audit journal 
75 in greater detail. As mentioned above, data diction- 
ary manager 85 is responsible for interrogating audit 
journal 75 from time to time and ensuring that 
changes to system objects are reflected in data dic- 
tionary 90. The interrogation and updating can be 
performed at predefined intervals, or when triggered 
by requests for data dictionary information, or on 
some other basis. The choice of lag time between the 
occurrence of a change to a system object and the re- 
flection of that change in data dictionary 90 is an im- 
plementation decision. Some implementations may 
not require that changes be immediately reflected in 
data dictionary 90. However, data dictionary 90 of the 
preferred embodiment is "closely coupled" to the ac- 
tual state of the system's objects. This close coupling 
is achieved through the use of synchronization func- 
tion 110, RQST data line 115, and reply queue 120. 
Synchronization function 110 of dictionary services 
45 controls the rate at which data dictionary manager 
85 queries audit journal 75. The WAIT portion of dic- 
tionary manager 85 initiates audit journal query activ- 
ity whenever a request is received on RQST data line 
1 1 5 or whenever a user defined time-out expires. The 



request that is sent by synchronization function 110 
can be initiated by a user or by an application pro- 
gram. Data dictionary manager 85 responds to the re- 
quest by processing all pending journal entries. When 

5 all of the journal entries have been applied to data dic- 
tionary 90, the user or application program is in- 
formed of such through the use of reply data queue 
1 20. Since the system administrator is responsible for 
both the frequency of requests and data dictionary 

10 manager 85's time-out value, the degree of coupling 
can be tailored to best fit system needs. 

Checkpoint line 130 is used by data dictionary 
manager 85 to mark the most recently processed 
journal entry and to provide for recovery in the event 

15 of a system failure. When data dictionary manager 85 
has exhausted the contents of audit journal 75, it logs 
its own checkpoint entry in audit journal 75 and enters 
a wait state. When dictionary manager 85 awakens, 
either through user stimulus 115 or through a time- 

20 out, it will eventually return its checkpoint entry. At 
this point, data dictionary manager '85 knows that it 
has effectively marked the last entry processed (i.e., 
the entry that was logged immediately before the 
checkpoint entry). To validate this fact, data diction- 

25 ary manager 85 will log a commit entry into the audit 
journal. Data dictionary manager 85 then knows that 
when recovering from a system failure it must locate 
the last commit entry/checkpoint entry pair whenever 
it resumes journal processing. Data dictionary man- 

30 ager 85 further knows that it must begin with the en- 
try that is logged immediately after the checkpoint en- 
try to ensure that it processes any entries that were 
logged between the checkpoint entry and the commit 
entry. It is important to note that since a system failure 

35 could take place at any time, there may be "unpaired" 
checkpoint entries within audit journal 75. As men- 
tioned above, dictionary manager 85 knows that it 
must find a commit entry/checkpoint entry pair to be 
certain that all of the journal entries have been proc- 

40 essed. This means that dictionary services 45 must 
be able to tolerate an entry being processed process- 
ed more than once. 

It is in this manner that data dictionary manager 
85 is able to keep data dictionary 90 up to date with 

45 the status of system objects even over a system re- 
start 

Fig. 4 shows a flow diagram of the activities of 
data dictionary manager 85. At startup, data diction- 
ary manager 85 will find its starting point by locating 

so the last commit entry/checkpoint entry pair 140. 
Since a commit entry is not logged until data diction- 
ary manager 85 is certain that it has accurately 
marked its ending point with a checkpoint entry, data 
dictionary manager 85 knows that the first journal en- 

55 try that requires processing will be the first journal en- 
try that follows the checkpoint entry. Once the posi- 
tion is located, data dictionary manager 85, after at- 
tempting to retrieve a journal entry 145, first asks 
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whether a journal entry exists 150. If the answer is 
"no", data dictionary manager 85 first asks whether 
the last entry returned was a commit entry 192. This 
latter test is in place to avoid placing multiple check- 
point entries into audit journal 75 during idle time. If 5 
the answer to the test is no, dictionary manager 85 
logs a checkpoint entry in audit journal 75 in step 200. 
In either case, data dictionary manager 85 will next 
check for any pending synchronization requests from 
dictionary services 45 in step 220. If there are syn- 10 
chronization requests pending, data dictionary man- 
ager 85 will respond 205 to the request via reply data 
queue 120. In either case, data dictionary manager 
85 will eventually reach decision block 210. Here, 
data dictionary manager 85 will either proceed to wait 1 5 
state 225 or terminate 215. The choice depends on 
whether a system shutdown has occurred. In the pre- 
ferred embodiment, the shutdown information is ac- 
cessed via a system variable. 

Referring back to the NO option of decision block 20 
150, data dictionary manager 85 will next ask whether 
the journal entry indicates that a significant change 
to the configuration of journaling mechanism 38 has 
taken place 155. An example of such a change would 
be the re-configuration of journaling mechanism 38 to 25 
no longer record create activity. If this were to take 
place, data dictionary manager 85 could no longer 
keep data dictionary 90 up to date with the state of the 
system's objects. Therefore, data dictionary manager 
85 would inform the system administrator of such 195 30 
and terminate 190. If the journal entry does not indi- 
cate such a change in configuration, data dictionary 
manager 85 will ask whether the journal entry is dic- 
tionary related 160. The following journal entires of 
Fig. 2C are dictionary related: SV (system values), 35 
DO (deletes), CO (creates), OM (move/rename), OR 
(restore), CK (checkpoint), CT (commit). Since not all 
entries in audit journal 75 are dictionary related, data 
dictionary manager 85 must be able to ignore non-dic- 
tionary related journal entries. Hence, the process 40 
will return to block 145 whenever a non-dictionary re- 
lated journal entry is detected. 

If it is determined that the journal entry is diction- 
ary related, data dictionary manager 85 then deter- 
mines whether the entry is a checkpoint journal entry 45 
or a commit journal entry 185. If the journal entry is 
neither, data dictionary manager 85 proceeds to up- 
date data dictionary 90, block 180. For example, if a 
create entry (CO) was encountered, data dictionary 
manager 85 would use the data stored in the entry to 50 
apprise data dictionary 90 of the existence and nature 
of the new object The process then repeats. If, in- 
stead, data dictionary manager 85 determines that 
the journal entry is either a checkpoint entry or a com- 
mit entry, it will, depending on exactly what type of an 55 
entry it is, either log a commit entry 175 or simply at- 
tempt to retrieve another journal entry 145. 

Although the preferred embodiment of the sub- 



ject invention uses the system's existing audit journal, 
other ways of storing the information are also possi- 
ble. For example, the operating system could main- 
tain a separate journal for use by the data dictionary 
manager. Indeed, virtually any means for storing in- 
formation about changes to system, objects could be 
used to provide the feedback capability of the present 
invention. 



Claims 

1. A method for maintaining a synchronized data 
dictionary comprising the steps of: 
accessing objects coordinated by said dictionary; 
effecting changes upon said objects; 
recording information about said changes in a re- 
cord separate from said data dictionary; and 
updating said data dictionary to reflect said 
changes. 

2. The method of claim 1 wherein said recording 
step comprises the step of placing an entry in an 
audit journal for each of said changes. 

3. The method of claim 2 wherein said updating step 
comprises the steps of: 

retrieving said information about said changes 
from said audit journal and ensuring that said in- 
formation is reflected by said data dictionary. 

4. The method of claim 2 for maintaining said data 
dictionary across a system failure, said method 
comprising the steps of: 

marking said audit journal with a first mark to in- 
dicate the location of a last entry processed; 
validating said last entry processed with a sec- 
ond mark; 

returning to the first instance of said first mark 
that has been validated by said second mark; and 
updating said data dictionary accordingly. 

5. The method of claim 4 wherein said marking step 
comprises the step of using a checkpoint entry. 

6. The method of claim 4 wherein said validating 
step comprises the step of using a commit entry. 

7. A method for adjusting the synchronization of a 
data dictionary, said method comprising the 
steps of: 

recording information about changes to system 
objects in an audit journal, where said informa- 
tion is in the form of one entry per change; 
accepting as prompts synchronization requests, 
where said synchronization requests are re- 
quests to process said entries of said audit jour- 
nal; 
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waking whenever an adjustable timeout occurs; 
and 

responding to said synchronization requests and 
said timeout by retrieving said entries from said 
audit journal and updating said data dictionary 
accordingly. 

8. A data dictionary manager comprising : 

means for accessing objects coordinated by said 
dictionary; 

means for effecting changes upon said objects; 
means for recording information about said 
changes in a record separate from said data dic- 
tionary; and 

means for updating said data dictionary to reflect 
said changes. 



10 



15 



change; 

means for accepting as prompts synchronization 
requests, where said synchronization requests 
are requests to process said entries of said audit 
journal; 

means for waking whenever an adjustable time- 
out occurs; and 

means for responding to said synchronization re- 
quests and said timeout by retrieving said entries 
from said audit journal and updating said data dic- 
tionary accordingly. 



The data dictionary manager of claim 15 wherein 
said means for recording comprises means for 
placing an entry in an audit journal for each of 
said changes. 



20 



10. The data dictionary manager of claim 16 wherein 
said means for updating comprises: 

means for retrieving said information about said 25 
changes from said audit journal; and 
means for ensuring that said information is re- 
flected by said data dictionary. 

11. The data dictionary manager of claim 9 for main- 30 
taining the data dictionary's integrity across a 
system failure, said data dictionary manager fur- 
ther comprising: 

means for marking an audit journal with a first 
mark to indicate the location of a last entry proc- 35 
essed; 

means for validating said last entry processed 
with a second mark; 

means for returning to the first instance of a said 
first mark that has been validated by a said sec- 40 
ond mark; and 

means for updating said data dictionary accord- 
ingly. 

12. The data dictionary manager of claim 11 wherein 45 
said means for marking comprises a checkpoint 
entry. 

13. The data dictionary manager of claim 11 wherein 

said means for validating comprises a commit en- so 
try. 

14. A data dictionary manager which allows for the 
adjusting of the synchronization of a data diction- 
ary, said data dictionary manager comprising: 55 
means for recording information about changes 

to system objects in an audit journal, where said 
information is in the form of one entry per 
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